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Abstract 

The Mission Operations Directorate (MOD) at the Johnson Space Center (JSC) has put in place a 
Model Based Systems Engineering (MBSE) technological framework for the development and 
execution of the Flight Production Process (FPP). This framework has provided much added 
value and return on investment to date. This paper describes a vision for a model based 
Decision Support System (DSS) for the development and execution of the FPP and its design and 
development process. The envisioned system extends the existing MBSE methodology and 
technological framework which is currently in use. 

The MBSE technological framework currently in place enables the systematic collection and 
integration of data required for building an FPP model for a diverse set of missions. This 
framework includes the technology, people and processes required for rapid development of 
architectural artifacts. It is used to build a feasible FPP model for the first flight of spacecraft 
and for recurrent flights throughout the life of the program. This model greatly enhances our 
ability to effectively engage with a new customer. It provides a preliminary work breakdown 
structure, data flow information and a master schedule based on its existing knowledge base. 
These artifacts are then refined and iterated upon with the customer for the development of a 
robust end-to-end, high-level integrated master schedule and its associated dependencies. 

The vision is to enhance this framework to enable its application for uncertainty management, 
decision support and optimization of the design and execution of the FPP by the program. 
Furthermore, this enhanced framework will enable the agile response and redesign of the FPP 
based on observed system behavior. The discrepancy of the anticipated system behavior and the 
observed behavior may be due to the processing of tasks internally, or due to external factors 
such as changes in program requirements or conditions associated with other organizations that 
are outside of MOD. The paper provides a roadmap for the three increments of this vision. 
These increments include (1) hardware and software system components and interfaces with the 
NASA ground system, (2) uncertainty management and (3) re-planning and automated 


execution. Each of these increments provides value independently; but some may also enable 
building of a subsequent increment. 


1. First Increment: Background 


A language for the capture, representation and storage of architecture and design information for 
the FPP has been developed, established and is currently in use. This language is the basis for an 
ontology that has constituted the schema for an SQL database, the Mission Operations 
Directorate Enterprise Architecture Repository (MODEAR). MODEAR is in operation and is 
currently providing support for upcoming missions. This has enabled the formalization of the 
FPP and the collection and organization of the expert data which is typically used for a mission 
design. The process flow diagrams associated with each of the key functions within the FPP are 
generated and updated in Visio by the Subject Matter Experts (SMEs). These process flow 
diagrams are the primary means for the SMEs to convey architecture information to MODEAR. 
Using helper applications, MODEAR works interactively with Microsoft Visio to, one hand, 
provide the latest repository data to experts working on the Visio diagrams and on the other 
hand, conduct consistency checking on the new data created by them before uploading to the 
database. MODEAR facilitates the export of data associated with architectural artifacts to 
customized architecture development tools. Details of the FPP Re-engineering project that has 
created the technological framework referred to are included in [6], 


Process Flows / 
Product Dependencies 


Arf'hitprtura Fv^min^tinn 


D; Propose 

' Visi o Datagram MLfXML) 



•Visio Drawing object for CM 
• Datagram ML-to-( MO DE AR j XML transform 


Repository $ W Processor 
•Verify against architecture baseline 
•CM; Staging & Approval 


Fjfecu table Architecture 

APPEAR XML py 

| Discrete Event Simulation 

1 

DBS Parameters I 


Process Model 



g 



Gurttlcn 

10 




JHL 

Timing 

L4HHMMSS 


M j j 


4? 

Probability 

76% 





Triggers 

VUnn [land ()thtn 
0 


Figure 1: Process used for generating architectural and design artifacts 











In the post-Constellation era, MOD is collaborating with a host of new customers. The artifacts 
established by the FPP Re-engineering project in support of the Constellation Program (CxP) 
provided a preliminary Work Breakdown Structure (WBS), data flow information and a master 
schedule based on the existing knowledge base in MODEAR. This knowledge base has been 
leveraged by refactoring the PFDs that had previously been generated for the CxP [7], These 
refactored PFDs are recursively applied in the construction of what is called a Design Reference 
Mission (DRM). The DRM includes a general set of functions and their associated, instanced 
PFDs and inputs/outputs. It is meant to serve as a reference and a starting point to the 
development of concrete point designs for specific missions. 

The current approach for designing an FPP using the existing infrastructure is for the SMEs to 
each sustain their associated PFDs. The data is integrated and the overarching FPP model is 
managed by members of the Special Analysis Team (SAT). The SAT team comprises SME 
members with expertise in each of the domains, Systems Engineers who conduct the integration, 
and architectural modeling and analysis experts. The SAT team holds a meeting periodically to 
review and comment on the state of the model. Based on SME coordination and discussions 
conducted through the SAT forum, SMEs may refine their associated PFDs. The FPP model 
matures through this process of SME coordination and integration. 

The current process used for generating architectural and design artifacts using this framework is 
depicted in figure 1. The architecture team, which is also called the Special Analysis Team 
(SAT), first defines the general scope of the activity and PFD’s that need to be customized for 
the study in question (A. Define). These PFDs are then checked out of MODEAR (B. Checkout) 
into the Visio application to be composed ore refined by the SME’s that represent the divisions 
within MOD (C. Compose). The SME’s then propose changes and customizations to the PFD’s 
(D. Propose). Their proposals are checked in to MODEAR for processing. The process includes 
automated consistency checking to ensure that they satisfy the baseline rules for the architecture, 
and depending on the sensitivity of the changes that have been made, may also include an 
approval cycle in which the Lead System Architect approves the changes before it is updated to 
the MODEAR repository (E. Baseline). Once this data has been collected into MODEAR, it is 
described using architectural artifacts (F. Describe) and further validated and examined using 
executable models such as a simulation (G. Simulation). This process is then iterated until a 
satisfactory design is obtained. 

The vision is to extend the existing framework so that it may provide a more complete suite of 
tools and techniques for the purpose of supporting the decision making by Project Managers and 
of aiding in design optimization by System Engineers. An optimal design is one which satisfies 
the design criteria on schedule, while incurring minimum cost and risk. This suite would include 
modules for uncertainty management, decision support, and automated execution and re- 
planning. While each of the increments described below are in and of themselves beneficial, 
there are also certain dependencies between them. The second increment enhances the third 
increment and enables the fourth increment. 



2. Second Increment: MOD System Components and the NASA Ground 

System 

MOD at JSC is part of the NAS A- wide ground systems operation; the FPP has been focused on 
the operations pertinent to MOD. The hardware and software which is required for pre-mission 
planning, training and mission execution is distributed across MOD, the agency and external 
entities. While the current ontology and data structure associated with MODEAR is consistent 
with the Department of Defense Architectural Framework (DoDAF), it includes only the 
workflow related to the operations viewpoint and data flow processes, as well as some level of 
mapping between these and the system components used for their performance. It does not 
currently include details about the hardware and software system components themselves. 
Therefore, the architectural and modeling artifacts obtained from the data in MODEAR are based 
on process modeling and operational view development. While this in itself has resulted in a 
significant re-engineering of the FPP, it does not include the re-engineering of the system 
components used for executing the FPP tasks. Up to this point, the efficiency obtained from this 
effort has been due to the elimination of the process and product redundancies in the system that 
has been accomplished by the formalization of these processes. 

Nonetheless, visibility into the system level elements of the hardware and software system 
elements and interfaces that is used for performing the tasks and managing these resources 
further helps to enhance productivity and optimizes the system for the metrics of interest, such as 
cost, reliability and time to complete. Therefore the next immediate step involved in this effort 
is the extension of the MODEAR data model to allow for defining the system level architectural 
artifacts, the capture of the data associated with these elements and their connection to the tasks 
being performed. 

The ontology that has been designed for the FPP must also interface and interact with 
representations associated with the rest of the MOD as well as the NASA-wide ground system. 
The next generation Mission Control Center (MCC) at JSC, which includes some of the 
hardware and software used for the generation of the flight products, and which is responsible for 
the generation and execution of commands to the spacecraft, currently has a variety of 
requirements, design documentation and system modeling associated with it. Other elements of 
the human spaceflight operations (including the Spacecraft System Avionics, Communications 
System, User Applications, etc.) each have their own associated system models as well. 
Therefore, there exists a distributed data structure associated with the various ground systems for 
spaceflight operations; and the intricate dependencies between the various elements are being 
designed and developed. Figure 2 is a notional depiction of some of the various pieces of a 
distributed ground system architecture, including the models and data repositories. 

MODEAR has been designed with consideration toward a need to interface with other models 
and applications, and the inclusion of system level containers within its database will enable this 
capability. The output of the MODEAR database is defined in a standardized XML format and 



is readily transformable to other tools and applications. Furthermore, data from other sources 
can be digested into MODEAR by conducting appropriate transformations as well. 



Figure 2: Modeling and data management for the Ground System 

The completion of the second increment would enable the development of a more efficient FPP 
system altogether and the integration of the FPP with the rest of the MCC and the NASA ground 
system. 

3. Third Increment: Uncertainty Management 

The third increment provides the capability of conducting risk analysis. The use of risk-based 
design methodologies from early phase design is not a new concept at NASA. A good example 
of one such design is the Altair lunar Lander [9]. An interesting feature of this design is that it 
does also consider the risks associated with the human operators of the system during the design 
cycle and thus the systems level risk models include the hardware, software and human elements 
that contribute to risks. 

While it is possible to simply build risk assessment models based on the workflow processes and 
tasks that are used to perform them, a more complete model would also include the hardware and 
software components of the system that are used for executing the tasks and their configuration. 
Therefore the third increment can even now be added to the existing background work, but in 
order for it to be more complete and at the system level, it needs the establishment of the second 
increment. Alternatively, the second and third increments could be added in parallel. 

The design of the FPP is purely based on the elicitation and integration of Subject Matter 
Expertise (SME). The SME’s considered the “best-case” scenario when estimating the man- 
hours necessary for performing the tasks as well as the sequences of tasks performed. This was 
referred to as the “happy-path”. The next logical step is to identify the non-happy paths, and the 
things that may go wrong, or risks and develop strategies for mitigating these risks. 

Broadly, the uncertainty may be due to the following reasons: 


► Inaccuracies in the design 



SME’s might have left out activities during the design process. 


° System design without proper consideration of the operations profile and 
activities for the system. 

° Their estimate of time and resources for the performance of activities may be 
inaccurate. 

► Changes in requirements 

° Requirements for the mission in question may change and result in changes to the 
FPP process. 

► Resource availability 

° Resources such as humans, hardware or software may not be 100% available. 
This in itself may cause a delay in the completion of associated activities. 

► Errors in performing tasks 

Our initial suggested approach for managing these uncertainties is to build Probabilistic Risk 
Assessment (PRA) models for each of the Process Flow Diagrams (PFDs) on the critical path. 
According to the Pareto Principal, roughly 80% of the uncertainty is due to roughly 20% of the 
processes [7]. The idea here is that those 20% are the processes on the critical path. Once these 
PRA models are built, the team will deliberate on the causes of delay or error for these PFDs. 
Failure data from multiple sources, such as the Human Reliability data banks from the Nuclear 
Industry and failure records for the various hardware and software components of the MOD FPP 
system are then used to populate and execute these models. Execution of these models will give 
insight into areas of high risk and provide guidance for margin allocation. Allocating additional 
margins to the sensitive areas of the system will in turn help to mitigate the risks associated with 
them. 

For demonstration purposes, we consider a sample PFD which includes the sequence of 
activities: Change Request (CR) Generation, Book Manager Review & Approval, MOD Internal 
CR review. Figure 3 shows the Event Sequence Diagram (ESD) associated with this PFD. Note 
that this particular PFD represents the second set of the function “4.3.3_2: Conduct Flight 
Operations Reviews”. Therefore in the PRA model, it feeds into the third event tree (ET3) which 
represents the third step of this function. This dependency is shown via the “»ET3” notation in 
the tool. For each of the activities involved, it’s likely that it is conducted correctly and on time, 
or that there are delays or errors involved. Therefore at each event node, the tree has two 
branches. The upper branch indicates the success of the event and the lower branch its failure. 
The final tree has eight different possible outcomes since there are three events that each has two 
outcomes. In this case, if one activity is slightly delayed the final products are still generated 
correctly and on time, but if two activities are flawed there is a delay and if three are flawed, the 



final outcome is erroneous products. Now each of these activities involves multiple failure 
modes. So in order to assess the likelihood that each of these activities will be successful or 
erroneous, our PRA model includes a fault tree associated with them. Figure 4 shows a fault tree 
associated with the activity “Book Manager Review & Approval”. As it can be seen, the book 
manager may omit a step involved in the process, or there may be an existing error that is not 
caught as he reviews the document. The likelihood of these errors is obtained from the data 
banks associated with human errors from the Nuclear Industry [8]. The final probability of 
failure associated with each of the events is indicated in the Q variable which is represented in 
the ESD in figure 3 as well. The probability of each of the paths occurring is indicated in the 
frequency column at the left. 
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Figure 3: Sample Event Sequence Diagram for the PFD “Conduct Flight Operations Reviews” 
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Figure 4: Sample Fault Tree associated with the event “Book Manager Review & Approval” 

Another significant uncertainty involved is the estimate of the costs involved. The suggested 
approach for cost modeling is to map the elements of the functional breakdown for the generic 
FPP project into the WBS elements of projects for which there are available cost data and use 
that as a basis for cost estimation. While this result provides an estimate for the labor costs 
associated with the FPP, the facility costs can be derived by using the operational profile of the 
facility components that are generated from the simulation analysis. 

Some of the possible outputs of the cost modeling effort include: 

• An estimate for the labor costs associated with the development of the FPP for the project 
in question. 

o Uncertainty distribution for the estimate. 

• A list of the key cost drivers for the FPP. 

• An estimate for the facility costs associated with the FPP. 

o Uncertainty distribution for the estimate. 

Therefore when the third increment has been completed, the iteration process with the SAT team 
will include brainstorming about the risks and sensitivities of the system. This information will 
be collected and included in a PRA model and a cost model by a modeling expert on the SAT 
team and the results of the PRA/Cost-risk model will be deliberated upon and iterated in order to 
develop optimal margin allocation strategies. The main difference is that there will be a risk and 
cost-risk assessment module in the loop and instead of just one baseline design, there will be 
several alternatives, each with a specific risk and cost profile. 


4. Fourth Increment: Re-Planning and Automated Execution 

The fourth increment enables the semi-automated re-planning of the initial FPP design as the 
actual system behavior is observed and its corresponding data collected. This increment is 





dependent on the second implement and complimentary to the third increment. In other words, it 
is necessary to have information about the system hardware and software components within the 
models in order to conduct automated re-planning and execution. Furthermore, re-planning, in 
particular is an approach for managing uncertainties and hence complimentary to the third 
increment. 

Risk based margin allocation was suggested as an initial approach for managing uncertainty. 
This is due to the fact that additional margins allocated to sensitive areas of the design provide 
for the resources needed for re-planning. In other words, when there are errors or inaccuracies in 
the design, or when the requirements change, there is a need for the team to conduct a re-plan 
and re-design the template accordingly. Currently, this is done in a completely manual way. In 
other words, the team actually has to convene, discuss the issue being addressed and form a plan 
for changing the design to accommodate it. Nonetheless, this re -planning process can be semi or 
mostly automated in time. Much of the knowledge that the SAT team uses for the re-planning 
process has been captured within MODEAR. Formalizing this data and representing it in a 
format that would be amenable to automated optimization is the first step towards automated re- 
planning. The next step would be the development of heuristics and use of planning algorithms 
for the creation of the re-planning engine. 

The fourth increment calls for the development of an automated re-Planner as well as an 
execution engine. These two can be developed in parallel. 

During the implementation of an FPP process, the tasks are triggered in order. Each task is 
identified by the resources required for its’ processing and the prerequisites for it to begin. 
When the prerequisites for it have been met (i.e. all tasks on which it depends have already been 
processed), and the resources required for its execution are available, an automated executor or 
what we call an MLNE(Management Level Network Executive) triggers the application (or 
human expert) required to perform the task to begin processing. 

The status of the task in question then goes from “idle” to “in implementation”. Further, the 
exact time and resource that has been allocated to it will also show in its’ status. When the task 
has completed, it will check back into the system with its final status (completed, time-stamped) 
and the process continues. 

Throughout the implementation process, the actual measures of performance for the system are 
collected on a regular basis by synthesizing the data that is created from the execution of tasks. 
So, for instance, for each task, the time it takes to perform it and the amount of resource usage is 
recorded. As the system servers more and more FPP processes, a data set associated with each 
task is created and this data set is used to update the prior information that is collected from the 
SME’s regarding the tasks. 


Table 1 provides a list of the different types of uncertainty that could be dealt with using a Re- 
planning engine. 



Type of Uncertainty 

Type of Change 
Required 

Approach 

Level of 
Automation 

Requirement Change 

Design Change 

Re-planning 

Mostly automated 

Delta between the 
design versus the 
actual 

Resource allocation 
change. 

Margin 

management, Re- 
planning 

Mostly automated 

Flight Schedule 
Change 

Date changes 

Re -planning 

Mostly automated 

Personnel 

Availability 

Resource allocation 
change. 

Re-planning 

Mostly automated 

Resource Availability 

Resource allocation 
change. 

Re-planning 

Mostly automated 

Mission Profile 
Change 

Design Change 

Re-planning 

Semi-automated 

Other Change 
Requests 

Design or resource 
allocation changes 

Re-planning or 

Margin 

Management 

Semi-automated 


Table 1: Uncertainty Management Techniques. 


Once increment four has been completed, we have a Decision Support System that aids in the 
design, development, maintenance and management of the FPP. The selected baseline design 
will be executed semi-automatically with the commands going directly from the MLNE to the 
applications or humans responsible for performing the tasks. The metrics associated with the 
tasks will be collected and if there are discrepancies, the re -planner will develop a suggested re- 
plan for achieving the mission goals. The re-planner is analogous to the GPS system in a car. 
When it turns out that the path is different from the initially planned path, it finds an alternative 
path to the destination. The MLNE would then be analogous to a driver who can drive 
automatically for the most part but needs the actual human driver to supervise its performance 
and take charge as appropriate. Since the MLNE will be in communication with relevant 
applications, their status will also be automatically registered for the purpose of maintenance 
activities. 




5. DSS-FPP Elements and TAMU 


One of the key benefits of the approach established by the FPP Re-engineering project [6] is the 
fact that it facilitates MOD collaboration with other program elements and with other space 
agencies. This is achieved due to the systematic decomposition of the FPP and the visibility into 
the touch points of each of the modules or functions on which the architecture is built. The 
Transferable, Adaptable, Modular and Upgradeable [TAMU] features of this new paradigm are 
described in [5]. The Decision Support System described in the four increments above will 
further facilitate this collaboration by making the functions performed by the JSC based SAT 
team more efficient. These functions and the associated tools and approaches used for their 
conduct are summarized in table 2. 


Function 

Tool 

Approach 

Data Collection 

Process Flow Diagrams, Visio 

Visio Module - SME inputs 

Integration 

MODEAR 

MODEAR consistency checker/ integration 
approach 

Requirements Analysis 

MODEAR ++ 

Transfer requirements into structured 
ontology and integrate with the rest of the 
system 

Critical Path Analysis 

COTS tools/ Scheduling Application 

Standard CPM/ Scenario Analysis via 
Networks. 

Architecture Development 

COTS tool 

DODAF 

Optimization 

Planner/ To be developed 

A1 Planning Algorithms 

Synthesis 

SAT Team 

Team Consensus/ Discussions 


Re-Planner/ To be developed 

A1 Planning Algorithms 

Uncertainty Management 

Isograph Reliability Workbench 

Probabilistic Risk Assessment 

Execution 

MLNE (COTS tool) / To be 
developed 

Automated Execution 


Table 2: Key Functions performed by the FPP and their corresponding tool and approaches. 




6. Summary & Conclusions 

This paper explains the steps involved in extending a newly developed technological 
infrastructure into a full-on Decision Support System to support the development of Flight 
Products in a semi-automatic and optimal manner. The vision for this development reflects a 
developmental framework for MODEAR and the FPP model, which is divided into four 
increments, where the first increment has already been accomplished and is operational. It is 
important to point out that this first increment is already providing much added value and the 
expectation is for each of the remaining increments to have significant value and return on 
investment in and of themselves. Nonetheless, the way they have been designed, there is some 
level of dependency between these increments and each builds on the previous ones quite 
productively. 
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